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[Name of Document] SPECIFICATION 
[Title of the Invention] 

Robot Apparatus, Control Method Therefor, and Providing Medium 

[Claims] 
[Claim 1] 

A robot apparatus having an emotional model changed by an external factor or 
an internal factor, the apparatus comprising: 

detection means for detecting an external state; 
storage means for storing data; 

decision means for deciding an output of the emotional model based on a signal 
detected by the detection means; 

discrimination means for discriminating whether the output of the decision 
means is greater than a predetermined threshold value or not; and 

write means for writing data corresponding to the output in the storage means 
in the case where it is discriminated by the discrimination means that the output is 
greater than the predetermined threshold value. 
[Claim 2] 

The robot apparatus according to claim 1, further comprising read means for 
reading the data written by the write means. 
[Claim 3] 

The robot apparatus according to claim 1, further comprising rearrangement 
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means for rearranging the data written by the write means in accordance with the 
output of the emotional model. 
[Claim 4] 

A control method for a robot apparatus having an emotional model changed by 
an external factor or an internal factor, the method comprising: 
a detection step of detecting an external state; 
a storage step of storing data; 

a decision step of deciding an output of the emotional model based on a signal 
detected at the detection step; 

a discrimination step of discriminating whether the output at the decision step 
is greater than a predetermined threshold value or not; and 

a write step of writing data corresponding to the output in the case where it is 
discriminated at the discrimination step that the output is greater than the 
predetermined threshold value. 
[Claim 5] 

A providing medium for providing a program which causes a robot apparatus 
having an emotional model changed by an external factor or an internal factor, to 
execute processing comprising: 

a detection step of detecting an external state; 

a storage step of storing data; 

a decision step of deciding an output of the emotional model based on a signal 
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detected at the detection step; 

a discrimination step of discriminating whether the output at the decision step 
is greater than a predetermined threshold value or not; and 

a write step of writing data corresponding to the output in the case where it is 
discriminated at the discrimination step that the output is greater than the 
predetennined threshold value. 
[Detailed Description of the Invention] 
[0001] 

[Technical Field to Which the Invention Pertains] 

This invention relates to a robot apparatus, a control method therefor and a 
providing medium, and particularly to a robot apparatus, a control method therefor and 
a providing medium which enable intake and storage of external data in the case where 
an automatic mobile robot having an emotional model changed by an external factor 
and an internal factor is shifted to a predetermined state. 
[0002] 

[Prior Art] 

As an automatic mobile robot aimed at collecting marine data, an underwater 
search robot, for example, has been developed. For an underwater search robot, it is 
desirable to collect and record as many data difficult to collect as possible. These data 
include, as an example, seawater temperature data, sea stream data, depth data, 
terrestrial data and picture data. The operations of extracting and analysing effective 
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data is usually carried out after the underwater search robot has returned to the water 

surface. 

[0003] 

[Problem to be Solved by the Invention] 

However, in the automatic mobile robot, aimed at entertainment, protracted 
storage of meaningless data in the robot leads to increased memory costs. 
[0004] 

On the other hand, the operation of extracting effective data from the data 
stored in a robot is time- and labor-consuming. 
[0005] 

In view of the foregoing status of the art, it is an object of the present invention 
to enable extraction and recording of effective data alone in an automatic mobile robot. 
[0006] 

[Means to Solve the Problem] 

According to claim 1, there is provided a robot apparatus having an emotional 
model changed by an external factor or an internal factor, the apparatus comprising: 
detection means for detecting an external state; storage means for storing data; 
decision means for deciding an output of the emotional model based on a signal 
detected by the detection means; discrimination means for discriminating whether the 
output of the decision means is greater than a predetermined threshold value or not; 
and write means for writing data corresponding to the output in the storage means in 
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the case where it is discriminated by the discrimination means that the output is greater 

than the predetermined threshold value. 

[0007] 

According to claim 4, there is provided a control method for a robot apparatus 
having an emotional model changed by an external factor or an internal factor, the 
method comprising: a detection step of detecting an external state; a storage step of 
storing data; a decision step of deciding an output of the emotional model based on a 
signal detected at the detection step; a discrimination step of discriminating whether 
the output at the decision step is greater than a predetermined threshold value or not; 
and a write step of writing data corresponding to the output in the case where it is 
discriminated at the discrimination step that the output is greater than the 
predetermined threshold value. 
[0008] 

According to claim 5, there is provided a providing medium for providing a 
program which causes a robot apparatus having an emotional model changed by an 
external factor or an internal factor, to execute processing comprising: a detection step 
of detecting an external state; a storage step of storing data; a decision step of deciding 
an output of the emotional model based on a signal detected at the detection step; a 
discrimination step of discriminating whether the output at the decision step is greater 
than a predetermined threshold value or not; and a write step of writing data 
corresponding to the output in the case where it is discriminated at the discrimination 
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step that the output is greater than the predetermined threshold value. 
[0009] 

In the robot apparatus according to claim 1 , the detection means detects an 
external state and the storage means stores data. The decision means decides an 
output of the emotional model based on a signal detected by the detection means and 
the discrimination means discriminates whether the output of the decision means is 
greater than a predetermined threshold value or not. The write means writes data 
corresponding to the output in the storage means in the case where it is discriminated 
by the discrimination means that the output is greater than the predetermined threshold 
value. 
[0010] 

In the control method for a robot apparatus according to claim 4 and the 
providing medium according to claim 5, an external state is detected at the detection 
step and data is stored at the storage step. At the decision step, an output of the 
emotional model is decided based on a signal detected at the detection step, and at the 
discrimination step, it is discriminated whether the output at the decision step is greater 
than a predetermined threshold value or not. At the write step, data corresponding to 
the output is written in the case where it is discriminated at the discrimination step that 
the output is greater than the predetermined threshold value. 
[0011] 

[Mode for Carrying out the Invention] 
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A preferred embodiment of the present invention will now be described. To 
clarify the corresponding relation between each means of the present invention 
according to claims and the following embodiment, the features of the present 
invention are described, adding the corresponding embodiment (one example) inside 
parentheses subsequent to each means. However, this description is not intended to 
limit each means to the described embodiment. 
[0012] 

Specifically, the robot apparatus of claim 1 , having an emotional model changed 
by an external factor or an internal factor, includes: detection means (e.g., sensor 61 
of Fig.4) for detecting an external state; storage means (e.g., memory card 13 of Fig.2) 
for storing data; decision means (e.g., output selection unit 75 of Fig.4) for deciding 
an output of the emotional model based on a signal detected by the detection means; 
discrimination means (e.g., step SI of Fig.5) for discriminating whether the output of 
the decision means is greater than a predetermined threshold value or not; and write 
means (e.g., step S3 of Fig.5) for writing data corresponding to the output in the 
storage means in the case where it is discriminated by the discrimination means that 
the output is greater than the predetermined threshold value. 
[0013] 

The robot apparatus of claim 2 further includes read means (e.g., step SI 1 of 
Fig.7) for reading the data written by the write means. 
[0014] 
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The robot apparatus of claim 3 further includes rearrangement means (e.g., step 
S4 of Fig.5) for rearranging the data written by the write means in accordance with the 
output of the emotional model. 
[0015] 

Fig. 1 shows the appearance shape of a pet type robot 1 to which the present 
invention is applied. The pet type robot 1 is made up of legs 2a to 2d driven for 
movement, a head 3 housing a CCD (charge coupled device) video camera 1 1 (Fig.2), 
and a trunk 4. The pet type robot 1 is configured to walk autonomously in association 
with the inputs from variable sensors, such as a touch sensor 20 of Fig.2 as later 
explained, based on a program (emotional model) determining its own behavior. 
[0016] 

Fig.2 shows the internal electric configuration of the pet type robot 1. The 
picture data picked up by a CCD video camera 1 1 is sent to a signal processing unit 
12. This signal processing unit 12 processes the picture data routed from the CCD 
video camera 1 1 to memorize the picture data over an internal bus 18 into the memory 
card 13 or a DRAM (dynamic random access memory) 16. 
[0017] 

A CPU (central processing unit) 15 reads out the operating program stored in 
a flash ROM (read-only memory) 17 over the internal bus 18 to control the entire 
system. The operating program of the CPU 11, stored in the flash ROM 17, can be 
formulated or modified by an external personal computer (PC) 3 1 connected to the 
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signal processing unit 12. 
[0018] 

The signals detected by potentiometers 19a to 19d, a touch sensor 20 and a 
microphone 21 are routed through branching portions 24a to 24e to the signal 
processing unit 12, which signal processing unit 12 routes signals sent from the 
branching portions 24a to 24e over the internal bus 18 to the CPU 15. The CPU 15 
controls the operation of the actuators 22a to 22d and the legs 2a t o2d as well as the 

head 3 driven thereby, based on the supplied signals. The CPU 15 controls the speech 

outputted from the speaker 23. 

[0019] 

It is noted that the potentiometers 1 9a to 1 9c, touch sensor 20, microphone 2 1 , 
actuators 22a to 22d and speaker 23 constitute the legs, ears and mouth of the pet type 
robot 1 and are collectively tenned a CPC (configurable physical component) device. 
[0020] 

Fig. 3 shows an illustrative structure of the signal processing unit 12. 

[0021] 

A DRAM interface 41, a host interface 42 and a ROM interface 43 are 
connected to the DRAM 16, CPU 15 and to the flash ROM 17, while being connected 
to an external bus 44. A bus controller 45 controls the external bus 44, whilst a bus 
arbiter 46 arbitrates between the external bus 44 and an internal bus 47. 
[0022] 
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To a parallel port 48 and a serial port 50 is connected a personal computer (PC) 
31 as an external development environment. A battery manager 49 manages the 
residual capacity of a battery, not shown. The parallel port 48, battery manager 49 and 
the serial port 50 are connected over a peripheral interface 53 to the internal bus 47. 
[0023] 

The CCD video camera 1 1 furnishes the pictured picture data to a filter bank 
FBK 56, which then thins out- supplied picture data to formulate picture data of 
variable resolutions. These picture data are routed over the internal bus 47 to a direct 
memory access (DMA) controller 5 1 . The DMA controller 5 1 transfers the furnished 
picture data to the DRAM 1 6 for storage therein. 
[0024] 

The DMA controller 5 1 causes the picture data stored in the DRAM 16 to be 
read out and routed to an IPE (inner product engine) 55. The IPE 55 executes pre-set 
calculations using the furnished picture data. The calculated results are transferred to 
the DRAM 16 in accordance with commands from the DMA controller 5 1 for storage 
therein. 
[0025] 

To a USB (universal serial bus) host controller 57 is connected a CPC device 
25, which CPC device 25 is made up of, for example, the potentiometers 19a to 19d, 
touch sensor 20, microphone 21, actuators 22a to 22d and the speaker 23. The 
speech data furnished from the CPC device 25 are furnished via the USB host 

10 



controller 57 to a DSP (digital signal processor) 52, which then executes pre-set 
processing on the furnished speech data. To the USB interface 58 is connected the 
personal computer (PC) 32 as an eternal developing environment. A timer 54 routes 
time information to respective components over the internal bus 47. 
[0026] 

A emotional model 64 of the pet type robot 1 is constructed as shown for 
example in Fig.4. 
[0027] 

The sensors 61 to 63 detect stimuli applied from outside, such as environment, 
to convert the stimuli into electrical signals, which are outputted. These electrical 
signals are sent to input evaluation units 71, 72. It is noted that the sensors 61 to 63 
are comprised not only of the potentiometers 19a to 19d, touch sensor 20, 
microphone 21, but of a speech recognition sensor and a picture color recognition 
sensor etc, and converts the actuations by the user in taking care of the robot 1 or the 
speech the or she enunciated into electrical signals, which are outputted. Outputs of 
the sensors 61 to 63 are routed to the input evaluation units 71, 72. 
[0028] 

The input evaluation unit 7 1 evaluates the electrical signals furnished from the 
sensors 61 to 63 to detect a pre-set emotion. This pre-set emotion may, for example, 
be the emotion of pleasure. The input evaluation unit 71 sends an evaluation value of 
the detected emotion to an emotion module 73. To the emotion module 73 is allocated 
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a pre-set emotion (e.g., pleasure) such that the emotion parameter is increased or 
decreased based on the evaluated emotion value furnished by the input evaluation unit 
7 1 . The emotion module 73 sends the emotion parameter to an output selection unit 
75. 

[0029] 

Similarly, the input evaluation unit 72 evaluates the electrical signals furnished 
from the sensors 61 to 63 to detect the pre-set emotion. The pre-set emotion here is, 
for example, the emotion of anger. The input evaluation unit 72 sends the detected 
evaluation value of the emotion to an emotion module 74. To the emotion module 74 
is allocated a pre-set emotion (e.g. , anger) such that the emotion parameter is increased 
or decreased based on the evaluated emotion value furnished by the input evaluation 
unit 72. The emotion module 74 sends the emotion parameter to the output selection 
unit 75. 
[0030] 

The output selection unit 75 checks whether or not the emotion parameter 
supplied from the emotion modules 73, 74 exceeds a pre-set threshold value, and 
outputs the emotion parameter exceeding the threshold value. If the two emotion 
parameters from the emotion modules 73, 74 exceed the threshold value, the output 
selection unit 75 selects a larger one to output the selected parameter. 
[0031] 

A behavior generator 65 converts the emotion supplied from the output 
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selection unit 75 into a command instructing a specified behavior to route the 
command to an output unit 66 while feeding the command back to an output 
evaluation unit 76. 
[0032] 

The output evaluation unit 76 evaluates the behavior supplied from the behavior 
generator 65 and, if the behavior is performed, the output evaluation unit 76 performs 
control to decrease the emotion parameter corresponding to the behavior. 
[0033] 

An output unit 66 makes an output consistent with a behavior command from 
the behavior generator 65. The output unit 66 issues an output of the pet type robot 
1 which then behaves in accordance with a behavior command from the behavior 
generator 65. That is, the output unit 66 is made up of the actuators 22a to 22d and 
the speaker 23 driving the components such as legs 2a t o2d, head 3 or the trunk 4, and 
drives pre-set actuators to turn the head 3 or issue a whining or meowing sound. 
[0034] 

In the following explanation, it is assumed that the "pleasure" and "anger" are 
allocated to the emotion modules 73, 74, respectively. It is also assumed that the 
sensors 61, 62 and 63 are a picture color recognizing sensor, a voice recognizing 
sensor and a touch sensor 20, respectively. 
[0035] 

When fed from the picture color recognizing sensor 61, voice recognizing 
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sensor 62 and from the touch sensor 63 with electrical signals associated with the 
"yellow" , electrical signals corresponding to a pre-set frequency, such as "re" and with 
electrical signals corresponding to the "caressing" state, respectively, the input 
evaluation unit 71 evaluates the respective signals to determine the evaluation value 
for "pleasure". The input evaluation unit 7 1 routes the evaluation value "pleasure" to 
the emotion module 73. The emotion module 73 increases the emotion parameter 
based on the evaluation value for "pleasure". The emotion parameter is routed to the 
output selection unit 75. 
[0036] 

When fed from the picture color recognizing sensor 61, voice recognizing 
sensor 62 and touch sensor 63 with electrical signals associated with the "red", 
electrical signals corresponding to a pre-set frequency, such as "fa" and with electrical 
signals corresponding to the "hitting" state, respectively, the input evaluation unit 72 
evaluates the respective signals to detennine the evaluation value for " anger". The 
input evaluation unit 72 routes the evaluation value " anger" to the emotion module 74. 
The emotion module 74 increases the emotion parameter based on the evaluation value 
for u anger". The emotion parameter is routed to the output selection unit 75. 
[0037] 

The output selection unit 75 checks whether or not the emotion parameter 
supplied from the emotion modules 73, 74 exceeds a pre-set threshold value. It is 
assumed here that the emotion "anger" exceeds a threshold value. 
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[0038] 

The behavior generator 65 converts the emotion parameter for "anger" supplied 
from the output selection unit 75 into a command instructing a specified behavior 
(barking) to route the command to the output unit 66 , while causing the command to 
be fed back to the output evaluation unit 76. 
[0039] 

The output unit 66 issues an output in accordance with a behavior command 
(barking) from the behavior generator 65. That is, the output unit 66 outputs the 
corresponding sound. The "anger" is released by the pet type robot 1 barking so that 
its emotion of "anger" is suppressed. In this consideration, the output evaluation unit 
76 decreases the emotion parameter of the emotion module 74. 
[0040] 

Meanwhile, the above-mentioned output of the emotional model 64 is the 
emotion parameter differentiated with respect to time. That is, the larger the variation 
in the emotion parameter, the larger becomes an output of the emotional model 64. For 
example, if the emotion parameter "anger" of the pet type robot 1 is of a larger 
magnitude, the emotion parameter "pleasure" is rapidly changed (increased) by the 
robot viewing the yellow ball it likes. In this case, the picture data captured from the 
CCD video camera 1 1 is verified by the pet type robot 1 as being valid picture data so 
that it is stored in memory means such as the memory card 13. 
[0041] 

15 



The processing for storing effective data into the memory card 13 in the pet 
type robot 1 is hereinafter explained with reference to the flowchart of Fig. 5. 
[0042] 

First, at step SI, the CPU 15 checks whether or not an output value of the 
emotional model 64 has exceeded a pre-set threshold. If it is decided at step SI that 
the output value of the emotional model 64 has not exceeded the pre-set threshold 
value, the CPU 15 reverts to step SI. If, at step SI, the output value of the emotional 
model 64 is found not to exceed the pre-set threshold, the CPU 1 5 advances to step S2. 
[0043] 

At step S2, the CPU 15 checks whether or not there is any vacant area in the 
memory card 13. If, at step S2, it is found that there is a vacant memory area, the CPU 
15 advances to step S3 to cause the picture data captured from the CCD video camera 
1 1 to be stored in the vacant area of the memory card 13. The CPU 15 then causes 
the time and date data and the emotion parameter, in association with the picture data, 
as the characteristic information of the picture data. 
[0044] 

At step S4, the CPU 15 re-arrays the picture data in the order of the decreasing 
magnitudes of the emotional model 64. The CPU 15 then reverts to step SI. That is, 
the memory area of the memory card 13 is made up of a header 81 memorizing the 
time and date data and the emotion parameter as the characteristic information and a 
picture data portion 82 memorizing the picture data, as shown in Fig.6. The CPU 15 
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sorts the picture data in the order of the decreasing magnitude of the emotion output. 
[0045] 

If it is found at step S2 that there is no vacant memory area, the CPU 15 
advances to step S5, where the CPU 1 5 checks whether or not the current output value 
of the emotional model 64 is larger than the smallest value of the emotion output 
accompanying the picture data memorized in the memory card 13. That is, the CPU 
15 checks whether or not the current output value is larger than the value of the 
emotion output arrayed at the lowermost row in Fig.6. If it is found at step S5 that the 
current output value is not larger or smaller than the smallest value of the memorized 
emotion output, the CPU 15 reverts to step SI. 
[0046] 

If it is found at step S 1 5 that the current output value is larger than the smallest 
value of the memorized emotion output, the CPU 15 advances to step S6 where the 
CPU 15 erases picture data corresponding to the smallest value of the emotion output. 
The CPU 15 then advances to step S3 to cause storage of the then prevailing emotion 
output. This causes the emotion output to be stored to the memory card 13 
sequentially in the order of the decreasing magnitude of the emotion output. 
[0047] 

The processing in case effective data stored in the storage means by the pet type 
robot 1 is read out by the personal computer 3 1 is hereinafter explained with reference 
to the flowchart shown in Fig. 7. 
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[0048] 

First, the user extracts the memory card 13 from the PC card slot 14 to load the 
memory card 13 in a card slot, not shown, in the personal computer 31. When the 
memory card 13 is loaded in the card slot, the CPU, not shown, enclosed in the 
personal computer 3 1 reads out at step SI 1 picture data stored in the memory card 13. 
(The picture data are read out in the order of the decreasing magnitudes of the emotion 
output.) 
[0049] 

At step 12, the CPU re-arrays the read-out picture data in the chronological 
order of the date and time data to proceed to step SI 3. At step SI 3, the CPU stores 
the re-arrayed picture data in a memory, not shown, to terminate the processing. 
[0050] 

This allows the user to read out picture data at any time on the personal 
computer 31. Therefore, the user can read out picture data to enjoy the picture data 
as an album recording the life of the pet type robot 1 . 
[0051] 

In the above-described embodiment, data is mainly stored on the memory card 
13 . This, however, is not limitative since data can be memorized in the DRAM 16. 
[0052] 

In this specification, the system refers to logical integration of a plurality of 
devices, regardless of whether the respective constitutent devices are provided in the 
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same casing or not. 
[0053] 

In this specification, the providing medium for supplying a computer program 
executing the above processing may be exemplified by a transmission medium on a 
network, such as Internet or digital satellite, in addition to the information; recording 
medium, such as a magnetic disc or a CD-ROM. 
[0054] 

[Effect of the Invention] 

As described above, according to the robot apparatus of claim 1, the control 
method for a robot apparatus of claim 4 and the providing medium of claim 5, since 
data corresponding to the output of the emotional model is stored in the case where it 
is discriminated that the output of the emotional model is greater than a predetermined 
threshold value, the cost required for the memory can be reduced. 
[Brief Description of the Drawings] 

Fig.l is an appearance view of a pet type robot 1 embodying the present 
invention. 

Fig. 2 is a diagram showing an innate electrical structure of the pet type robot 

1. 

Fig.3 is a diagram showing a detailed structure of a signal processing unit 12. 

Fig.4 is a diagram for illustrating the emotional model 64. 

Fig. 5 is a flowchart for illustrating the processing of memorizing effective data 
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onto a memory card 13. 

Fig.6 illustrates the storage structure of the memory card 13. 

Fig.7 is a flowchart for illustrating the operation of reading out image data 
stored in the memory card 13. 
[Description of the Numerals] 

1 pet type robot; 2a to 2d leg; 3 head; 4 trunk; 11 CCD video camera; 12 
signal processing unit; 13 memory card; 14 PC card slot; 15 CPU; 16 DRAM; 17 
flash ROM; 18 internal bus; 19a to 19c potentiometer; 20 touch sensor; 21 
microphone; 22a to 22d actuator; 23 speaker; 24a to 24e branching unit; 31 PC; 41 
DRAM I/F; 42 host I/D; 43 ROM I/F; 44 external bus; 45 bus controller; 46 bus 
arbiter; 47 internal bus; 48 parallel port; 49 battery manager; 50 serial port; 51 
DMA controller; 52 DSP; 53 peripheral I/F; 54 timer; 55 IPE; 56 FBK; 57 USB 
host controller; 58 USB I/F; 61 to 63 sensor; 64 emotional sensor; 65 behavior 
generator; 66 output unit; 71, 72 input evaluation unit; 73, 74 emotion module; 75 
output selection unit; 76 output evaluation unit; 81 header; 82 picture data portion 
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[Name of Document] ABSTRACT 

[Summary] 

[Task] 

To reduce the cost required for the memory. 
[Means for Solution] 

A CPU 1 5 determines an output of a emotional model based on signals supplied 
from a touch sensor 20. The CPU 15 also deciphers whether or not an output value 
of the emotional model exceeds a pre-set threshold value. If the CPU finds that the 
output value exceeds the pre-set threshold value, it verifies whether or not there is any 
vacant area in a memory card 13. If the CPU finds that there is any vacant area in a 
memory card 13, it causes the picture data captured from the CCD video camera 1 1 
to be stored in the vacant area in the memory card 13. At this time, the CPU 1 5 causes 
the time and date data and the emotion parameter in the memory card 13 in association 
with the picture data. The CPU 1 5 also re-arrays the picture data stored in the memory 
card 13 in the sequence of the decreasing magnitude of the emotional model output. 
[Selected Drawing] Fig.2 



[Document Name] Official Correction Data 

[Collected Document] Patent Application 
<Authorized Information ■ Additional Information> 
[Patent Applicant] 

[Identification Number] 000002 1 85 

[Address] 7-3 5, Kitashinagawa 6-chome, 

Shinagawa-ku, Tokyo, Japan 

[Name] Sony Coiporation 

[Agent] Applicant 

[Identification Number] 100082131 

[Address] Inamoto International Patent Firm 

GOWA Nishi-Shinjyuku BLDG. 
6F, 5-8, Nishi-Shinjyuku 7-chome, 
Shinjyku-ku, Tokyo, 
Japan 

[Name] Yoshio Inamoto 



Information of Record for Applicant 



Identification Number: [000002 1 85] 



1. Date of Change: 
[Reason of Change] 
[Address] 

[Name] 



August 30, 1990 
Registration 

7-35, Kitashinagawa 6-chome, 
Shinagawa-ku, Tokyo, Japan 
Sony Corporation 



